Measurement of implementation components ten years after a nationwide introduction of empirically supported programs – a pilot study

BACKGROUND Ten years after the nationwide dissemination of two evidence-based treatment programs, the status of the implementation components was evaluated in a cross-sectional study. The aim of the study was to pilot a standardized measure of implementation components by examining the factor structure, the reliabilities of the scores, and their association with implementation outcome variables. The aim was also to compare implementation profiles of the two evidence-based programs based on multi informant assessments. METHODS The 218 participants in the study were therapists, supervisors, and agency leaders working with Parent Management Training, the Oregon model (PMTO), and Multisystemic Therapy (MST) in Norway. Interviewers filled in an electronic version of the Implementation Components Questionnaire during a telephone interview. RESULTS The factor analysis of the eight one-dimensional subscales resulted in an individual clinical-level factor and an organizational system-level factor. Age, experience, and number of colleagues in the workplace were negatively correlated with positive ratings of the implementation process, but the number of colleagues working with the same program predicted positive ratings. MST and PMTO had different implementation profiles and therapists, supervisors, and managers evaluated some of the implementation drivers significantly differently. CONCLUSIONS The psychometric quality of the questionnaire was supported by measures of internal consistency, factor analyses of the implementation components, and the comparisons of implementation profiles between programs and respondent groups. A moderate, but consistent association in the expected direction was found with the implementation outcome variables.